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ABSTRACT 

Nine cooperating rule-based systems, collectively called 
AUTOCREW, were designed to automate functions and decisions 
associated with a combat aircraft's subsystems. The organization of 
tasks within each system is described; performance metrics were 
developed to evaluate the workload of each rule base, and to assess the 
cooperation between the rule-bases. 

Each AUTOCREW subsystem is composed of several expert 
systems that perform specific tasks. AUTOCREW’s NAVIGATOR 
was analyzed in detail to understand the difficulties involved in 
designing the system and to identify tools and methodologies that ease 
development. The NAVIGATOR determines optimal navigation strat- 
egies from a set of available sensors. A Navigation Sensor Manage- 
ment (NSM) expert system was systematically designed from Kalman 
Filter covariance data; four ground-based, a satellite-based, and two 
on-board INS-aiding sensors were modelled and simulated to aid an 
INS. The NSM Expert was developed using the Analysis of Variance 
(ANOVA) and the ID3 algorithm. Navigation strategy selection is 
based on an RSS position error decision metric, which is computed 
from the covariance data. Results show that the NSM Expert predicts 
position error correctly between 45% and 100% of the time for a 
specified navaid configuration and aircraft trajectory. The NSM Expert 
adapts to new situations, and provides reasonable estimates of hybrid 
performance. The systematic nature of the ANOVA/ID3 method makes 
it broadly applicable to expert system design when experimental or 
simulation data is available. 

INTRODUCTION 

The computational and symbolic processing requirements for pilot- 
aiding systems pose many problems, e.g., multi-task scheduling and 
intersystem cooperation [1,2]. Expert systems, which are computer 
programs usually developed in a symbolic processing language such as 
LISP or PROLOG, have emerged to solve difficult domain-specific 
problems [3]. The designer generally extracts heuristics and specific 
knowledge from domain experts. This information is used to 
formulate a knowledge base consisting of parameters and rules. An 
inference engine uses rules to conduct a search and to set parameters, 
thereby inferring knowledge about the problem domain. 

Problem identification in the flight environment and expert system 
development has led to the design and implementation of expert 
systems in the areas of navigation, emergency procedures, and air 
traffic control [4-6]. The introduction of multiple system concepts 
such as global blackboard architectures for information exchange 
between knowledge bases has been successful in aerospace 
implementations [2,7], However, methods in multiple knowledge- 
base development, systems integration, and ensemble prototyping need 
further research if complex systems are to be developed for flight 
domain operations or other problem areas. 

A logical task classification scheme is a key factor in the successful 
development of multiple rule-based systems. In this paper a logical 
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structure for providing assistance to the pilot of a military aircraft was 
patterned on the functions of a World War II bomber crew [8], Based 
on this model, an ensemble of nine cooperating rule-based systems 
called AUTOCREW [9,10] was developed. Each system figuratively 
emulates a crew member's task responsibilities. 

To illustrate some of the issues involved in the design of an 
AUTOCREW subsystem, the problem of navigation sensor selection 
was studied in detail. With a large number of available navaid sensors, 
choosing an optimal or near-optimal sensor set becomes a large 
combinatorial problem. Convergence towards an optimal sensor 
configuration requires an exhaustive computer search utilizing simul- 
ation results as the basis for selection. In contrast, a small number of 
available navaids reduces the decision space considerably. Hence, a 
dilemma occurs; increasing sensor capability (and thus reliability and 
performance) increases decision-making complexity. 

A Navigation Sensor Management (NSM) Expen System that 
controls the selection of multi-sensor configurations was developed for 
AUTOCREW. Two on-board navigation aids (Doppler radar and Air 
data sensor) and five radio navigation systems (Global Positioning 
System, LORAN, TACAN, DME, and VOR) were modelled and 
covariance results obtained using a U-D implementation of the Kalman 
Filter equations [9], Up to three ground stations tind four satellites 
were simulated for typical high-performance, commercial jet transport, 
and general aviation aircraft trajectories. Mixed sensor simulations also 
were performed. These results provided the combinatorial, multi- 
factor source required for knowledge-base development 

The Analysis of Variance (ANOVA) statistical technique [13] was 
applied to the covariance results to identify the factors that cause 
variation in navigation performance. Once the important factors were 
identified, the relationships between them were determined. The ID 3 
algorithm [14], an inductive inference technique based-on the probabil- 
istic occurrence of events, was used to find these attribute relation- 
ships. Details of the development of the NSM Expert axe described in 
Ref. 9, and the main results are shown in this paper. 

AUTOCREW DESCRIPTION 

The nine AUTOCREW components [9,10] are COPILOT (flight 
control, aircraft performance, terrain following), ENGINEER (system 
diagnosis, reconfiguration, emergency procedures), NAVIGATOR 
(navigation sensor management, dynamic route planning), COMMUN- 
ICATOR (radio/daia operations), OBSERVER (lookout and alarm, 
identification friendl or foe) ATTACKER (offensive weapon control, 
target acqpisition/prioritization), DEFENDER (defensive weapon 
control), and SPOOFER (electronic measures/countermeasures). The 
ninth rule-based system, the EXECUTIVE, coordinates mission- 
specific tasks and has knowledge of the mission plan. In the 
AUTOCREW scheme, the human aircraft Pilot acts in the capacity of 
mission coordinator. The Pilot has full control of the aircraft and its 
on-board systems; AUTOCREW can provide automatic assistance 
when requested. In total, the AUTOCREW system performs 5 1 1 
tasks, is comprised of 406 rules, and uses 89 shared parameters [9]. 

Knowledge-Base Development of AUTOCREW Comp- 
onents 

Five main task groups are performed, by each AUTOCREW 
component: 1) tasks executed during an attack on the aircraft, (2) 

tasks executed during aircraft subsystem emergency or potential threat 




Figure 1. Knowledge Base of AUTOCREW 
Member NAVIGATOR 


situations, (3) additional tasks ordered by the EXECUTIVE, (4) tasks 
executed on a routine basis, and (5) mission-specific tasks that (for the 
most pan) are executed once and that require a high degree of 
cooperation among the various systems. Mission-specific tasks are 
divided into groups appropriate to each mission phase [10]. 

Figure 1 shows the graphical representation of AUTOCREWs 
NAVIGATOR loiowledge base (see Refs. 9 and 10 for details of the 
AUTOCREW knowledge bases). The NAVIGATOR'S tasks include 
navigation sensor management, navigation state and error estimation, 
and dynamic route planning. As seen in the figure, detection of an 
inbound enemy triggers the NAVIGATOR to perform defensive tasks. 
Rule NG4 shows that the NAVIGATOR finds the nearest friendly 
airfield able to starve the aircraft from its stewed airbase information. 


The NAVIGATOR also uses the OBSERVER'S information on 
friendly aircraft in the area and plans a recovery route. It is extremely 
important that the aircraft's location is known accurately in order to 
avoid such danger zones as Surface-to-Air Missile (SAM) belts. 
Tactical aircraft carry inertial navigation systems (INS) for high- 
frequency navigation information. A Kalman Filter can be used to 
integrate external navaids with an INS to estimate navigation errors. 
The imegreaed or hybrid INS gives accurate high-frequency navigation 
information with bounded errors in the state estimates (Rule N09). 
The error magnitudes depend on the accuracy of the navaid selected. 
The NAVIGATOR moniuss ground-based and satellite radio navaids 


174 


ORIGINAL PAGE 6S 
OF POOR QUALITY 
















Figure 2. AUTOCREW Workload - Mission-Specific Tasks and Emergency Scenarios 


in the area to determine if they are sufficiently operational for integra- 
tion purposes (Rule 1 1). From this list of navaids, the NAVIGATOR 
recommends the best combination meeting the computational resource 
constraints. The Pilot can change the NAVIGATOR'S selection if 
desired (Rule N14). 

The NAVIGATOR also is responsible for routinely monitoring 
flight plan progress. The NAVIGATOR first determines if the aircraft 
is closing on the target area. This information initiates additional tasks 
in the remaining AUTOCREW knowledge bases. The NAVIGATOR 
keeps a record of aircraft positions, restricted zones, and areas of 
weather disturbances as reported by the OBSERVER (Rule N18). The 
NAVIGATOR generates an alternate flight path for the following 
conditions: flight path conflicts, flight plan behind schedule, and fuel 
considerations. The EXECUTIVE is consulted when the 
NAVIGATOR recommends an alternate route. The EXECUTIVE 
considers the effect of the route change on the overall mission plan and 
aircraft's safety, and it makes a GO/NO GO recommendation (Rule 
N17). The Pilot can accept or reject the EXECUTIVE’S 
recommendation in Rule N19. If the new route is selected, the 
NAVIGATOR posts the route plan details on the display. As noted 
above, the COPILOT changes its steering commands in response to the 
route change. When the ATTACKER acquires enemy targets and the 
plan is to engage, the NAVIGATOR generates routes to each target, as 
seen in Rule N20. 

AUTOCREW Simulation Results 

Simulation and comparative workload results for two mission 
scenarios are given in this section. The scenarios are inbound SAM 
attack on the aircraft and Pilot incapacitation. Results are presented for 
five mission phases (routine, pre-flight, launch, attack, and recovery) 


and three emergency conditions (inbound weapon, enemy aircraft 
detected, and pilot incapacitated). 

AUTOCREW workloads for three emergency conditions were 
obtained and compared with the routine workload, referred to as 
"cruise mode" (Table I). For the three scenarios the workload 
progressively increases as the situation gets worst:. There is a dramatic 
increase in workload between cruise and attack modes with an inbound 
SAM, and there is an additional increase between the inbound SAM 
scenario (cruise), and inbound SAM with the Pilot incapacitated 
(cruise). The workload nearly doubles for an inbound SAM when the 
aircraft is already engaging another aircraft (attack mode). 

Table I Scenario Workload-Increase Comparisons 


Scenario 

Workload 

Increase, % 

Cruise Mode 

Tasks 

Decision-Mal 

Inbound SAM/Cruise Mode 

26.3 

28.1 

Inbound SAM & Pilot Incap- 
acitated/Cnnse Mode 

39.5 

38.6 

Inbound SAM/Attack Mode 

77.2 

75.2 


A comparison of the workload distribution among the AUTO- 
CREW members for each of the given mission phases is shown in Fig. 
2. The Rule Fraction is the ratio of number of rules fired during the 
mission phase for the specified AUTOCREW member to the total 
number of mission phase rules in all AUTOCREW knowledge bases 
[9], The Parameter Fraction is the ratio of number of tasks performed 
during the mission phase for the specified AUTO CREW component to 
the total number of mission phase tasks in all AUTOCREW knowledge 
bases. These metrics were used to characterize the knowledge base 



workloads. The ENGINEER, COMMUNICATOR, OBSERVER, 
NAVIGATOR and EXECUTIVE have the largest amount of roudne 
tasks to perform each search cycle. During the pre-flight phase, the 
ENGINEER, COMMUNICATOR, and NAVIGATOR are the busiest 
components, as each prepares the aircraft for takeoff. About 40% of all 
the launch phase tasks are performed by the COPILOT, as shown in 
Fig. 2, while the next largest task load is executed by the 
COMMUNICATOR and ENGINEER (each having 17% of the total 
launch workload). Most of the attack phase work is done by the 
ATTACKER (33%), whereas the recovery phase is dominated by the 
COPILOTS (35%) activities. Most of the workload during an inbound 
weapon attack is performed by the DEFENDER (about 30% in Fig. 2). 
The ATTACKER'S workload also increases at this time, as it monitors 
the DEFENDER'S firepower capability. 

There is evidence of more SPOOLER decision-making when an 
enemy aircraft is detected than when the OBSERVER detects an 
inbound weapon. This is due to its Electronic Counter Measures 
(ECM) strategy consultation with the EXECUTIVE. The COPILOTS 
workload also increases during these two emergency conditions; in 
both scenarios, the COPILOT selects an appropriate evasive maneuver. 
These selections are based on the aircraft capability information pro- 
vided by the ENGINEER. The EXECUTIVE'S workload also increas- 
es at this time to assist the Pilot in making decisions. When the Pilot is 
incapacitated, the EXECUTIVE becomes the primary decision-maker, 
as shown in Fig. 2. The ENGINEER'S major task during this scenario 
is to monitor the Pilot's condition, resulting in a 10% workload 
increase. 

DEVELOPMENT OF A NAVIGATION SENSOR 
MANAGEMENT EXPERT SYSTEM 

The remainder of this paper focuses on the issues involved in 
designing the navigation sensor selection expen in NAVIGATOR. 
This example demonstrates how simulation data and statistical analysis 
can be used to systematically design an expert system. 

To determine the performance of the various hybrid navaid combin- 
ations, the following U-D Kalman filter simulations were performed: 

1 . Single-type hybrids: GPS, LORAN, TACAN, DME, VOR, 

Doppler Radar, or Air Data Sensor aiding an INS 

2 . Number of stations used in a single-type hybrid 

3 . Multi-type hybrids: Combinations of different navaid types aiding 

an INS 

4. Aircraft trajectories simulated: High-performance, commercial, 

general aviation 

The root sum of squares (RSS) of the north and east component 
errors was selected for the hybrid system performance metric. The 
primary function of this expen system is to select the external navaid 
sensors that provide the smallest possible RSS position error from a 
large set of available sensors. The Analysis of Variance (ANOVA) 
technique [13] is used to identify the factors that make statistically 
significant contributions to the decision metric. Then, the ED3 
algorithm determines the relationships between these factors [13,14], 

Identifying Important Factors Using ANOVA 

The RSS position error histories from over 200 covariance simul- 
ations were obtained, and the data were used in an ANOVA four-factor 
navaid experiment. The goal of the experiment was to identify which 
of the factors (navaid type, number of ground stations, trajectory 
effects, performance history) and their interactions had statistically 
significant impatxs on the RSS position error. The factor values used 
in the ANOVA aeperiment were: 

Navaids=[ VOR, DME, LORAN, TACAN, GPS ) (Model 1) 

Number of Stations* (One, Two, Three) 

Trajectory Type*(High-Performance, Commercial 
Transport, General Aviation } 

Performance History * (Intervals: I, II, HI, IV j 

Since each trajectory consists of four, 15-minute legs, the Performance 
History (or "time interval") factor refers to the RSS error obtained 


within each 15-minute time frame. Four single-station, six double- 
station, and four triple-station hybrids were simulated using 
combinations of four different Stations (named Stations A-D) [9,12], 

The ANOVA technique was applied to the simulation results as 
follows: First, the mean value of the RSS position error and the 
variance using all simulation data were computed. The ANOVA model 
decomposes the variance into a sum of variances, each associated with 
a potentially contributing factor and factor interactions. For example, if 
a two-factor ANOVA experiment using navaid type and number of 
ground stations was performed, the total variation of the RSS position 
error in the complete set of simulation results would be: 

SStotal = SSnav+SSstat + SSnav-statint 

+ SSunexplained (1) 

Here, SStotal is the total variation in the RSS position error based on 
all simulations, SSnav is the variation in the RSS position error due to 
differences in navaid type, SSstat is the variation in the RSS position 
error due to the number of ground stations, and SSnav-stat int is the 
variation in the RSS position error due to interaction effects. 
SSuhexplained is the variation in the RSS position error that cannot be 
attributed to navaid type, the number of stations, or interactions 
between these two factors. 

Equation (1) was expanded for the four-factor navaid experiment 
defined by Model 1 [9,12]. Using the computed sums-of-squares, 
Scheffe multiple comparison tests were performed to identify the 
specific differences within the groups (e.g., RSS position error 
differences between different navaid types). 


Extracting Rules Using Induction: The ID3 Algorithm 

The ID3 Algorithm uses inductive inference to extract rules [14] 
from an example set. The problem space is descr.bed in terms of 
attributes, where each attribute is characterized by a set of values that 
define the possible "states." For example, in Model 1, the navaid type 
and number of ground stations were shown to be attributes affecting 
RSS position error. The ID3 algorithm defines the shape of the 
decision tree as it identifies the most important attribute at each decision 
node. The algorithm uses an Information-Theoretic Measure (ITM) 
that minimizes the number of tests (attribute nodes) necessary to define 
the decision tree. The ID3 algorithm uses the ITM in a splitting strategy 
[14] to deride which attribute provides the most information from the 
example sea (See Refs. 9, 12, 14 for details of the ID3 algorithm.) 


Developing the ID3 Attribute Framework Using ANOVA 
Results 

The Model 1 covariance simulations were used to extract decision 
trees for the NSM Expert system. Eleven attributes were defined for 
the ID3 framework: Navaid type, trajectory leg, aircraft groundspeed, 
number of ground stations, minimum geodetic distance from station, 
maximum geodetic distance from station, (Max - Min) distance on 
trajectory leg, minimum line-of-site angle from station, minimum line- 
of-sight angle from the station, direction of flight (approaching or 
receding) relative to station, and RSS position error class on previous 
trajectory leg. The ID3 algorithm determined how these attributes were 
related to each other and to the final RSS position error. 

The classification scheme chosen to represent the RSS position 
error endnode in the NSM derision trees is depicted in Table II. The 
velocity, distance, and line-of-sight angles were expressed in terms of 
ranges instead of individual values, so the expert system weights 
trends more heavily than specific examples. This makes the expert 
system more adaptable to new conditions, because matches between 
the actual and knowledge-base cases can be obtained more frequently. 
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Table II RSS Position Error Classification 
Scheme 


Accuracy 

[High] [Medium] [Low] 


Error 


Error 


Error 


Range, 

Code 

Range, 

Code 

Range, 

Code 

N. Mi. 


N. Mi. 


N. Mi. 


0.0-0.02 

c-1 

0.10-0.20 

c-6 

1.0-1. 5 

c-15 

0.02-0.04 

c-2 

0.20-0.30 

c-7 

1. 5-2.0 

c-16 

0.04-0.06 

c-3 

0.30-0.40 

c-8 

2.0-2. 5 

c-17 

0.06-0.08 

C-4 

0.40-0.50 

c-9 

2. 5-3.0 

c-18 

0.08-0.10 

c-5 

0.50-0.60 

c-10 

3. 0-3. 5 

c-19 



0.60-0.70 

c-1 1 

3. 5-4.0 

c-20 



0.70-0.80 

c-12 

4.0-4.5 

c-21 



0.80-0.90 

c-13 

4.5-5. 0 

c-22 



0.90-1.00 

c-14 

> 5.00 

c-23 


The example set was developed using the attribute framework 
described above. The RSS position errors for each simulation were 
classified on each trajectory leg using the scheme in Table II. The ID3 
example base was then created from each single-, double-, and triple- 
station simulation. 

NSM Decision Trees 

The NSM example set was divided into 17 smaller example sets. 
The GPS and on-board navaid examples were grouped into one expen, 
whereas the ground-based navaid examples were divided according to 
navaid type and time (15-minute intervals). The ID3 algorithm 
constructed decision trees for each of the 17 small expen systems to 
comprise the larger NSM Expen. The total number of examples used 
to develop the NSM Expen System was 932, based on 260 Kalman 
Filter covariance simulations. An additional 37 simulations were per- 
formed to obtain an RSS error-estimate decision tree for navaid-type 
combinations. The NSM expen system prompts the user for a set of 
flight conditions commensurate with the aitribute/vaiue lists used in the 


example set, and the resulting RSS classification code is returned to the 
user from the decision tree. 

Figure 3 shows decision trees for single-, double-, and triple- 
TACAN station combinations on the second 15-minute trajectory leg. 
Here, the majority of the testing nodes are trajectory parameters 
(distance, LOS angle, direction of flight with respect to the stations). 
The top or root node is the aircraft's direction of flight. This is 
expected because the distance and LOS angle attributes arc dependent 
on directional motion. Distance, LOS angle, and groundspeed are 
results of the aircraft's motion; hence, they represent more specific 
problem parameters, and it is expected that these parameters appear 
deeper in the decision tree. Figure 3 also shows that distance, ground 
velocity, LOS angle, and hybnd performance history are significant 
factors in RSS error prediction. 

The decision tree in Fig. 4 shows the expected position error range 
when different navaid types are integrated into a hybrid system. The 
RSS position errors for these simulations were averaged over the entire 
flight time for the high-performance trajectory. The tree is organized in 
terms of the navigation method used; (1) Distance-Velocity (p-V), (2) 
Bearing-Velocity (6-V), (3) Distance-Bearing (p-0), (4) Distance- 

Distance (p-p), (5) Bearing-Bearing (9-6), and (6) Velocity-Velocity 
(V-V). These results show that LORAN is a better distance-measuring 
navaid than DME and that Doppler Radar provides tetter navigation 
accuracy than the Air Data Sensor when p-V navigation is used. The 
p-6 results show that it is possible to obtain good performance when 
LORAN and VOR are used. The LORAN/DME hybrid gives better 
results than two DME stations but worse performance than two 
LORAN stations. By far the worst results are obtained using two VOR 
stations. 


Performance of NSM Expert on Test Trajectories 

Two high-performance trajectories were used in the performance 
evaluation of the NSM Expert. Multi-station hybrids were simulated 
on each test trajectory for the DME, VOR, TACAN, and LORAN 
systems. In total, 60 covariance simulations were performed for the 
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Decision Trees for Two Available 
TACAN Stations 


Decision Trees for Tliire© Available 
TACAN Stations 



Decision Trees Predicting RSS Position Error Range for an INS 
” ' “ ' the Second IS Minutes of Fli 


177 

ORIGINAL PAGE IS 
OF POOR QUALITY 














Figure 4. Decision Tree Predicting RSS Performance When 

Different Navaid Combinations are Used to Aid an INS 


two test trajectories. The results for each simulation were classified on 
each trajectory leg according to the scheme in Table II. The total num- 
ber of matches was counted on each trajectory leg for the seven navaid 
types studied. Figure 5 shows how well the NSM Expert predicts the 
RSS position error for each hybrid configuration. The predictive 
performance metric for each navaid is defined as the percentage of 
number of matches obtained from the total number of combinations 
tested for that navaid. 



Figure 5. Performance of Navaid Experts 
on Test Trajectories 

The NSM Expert performed very well on the two test trajectories. 
Figure 5 shows that the NSM Expert correctly predicts the RSS 
position error better than 70% of the time on test Trajectory #1. The 
system required only the trajectory information and its knowledge of 
hybrid system performance to make these predictions. However, its 
predictive capability on test Trajectory #2 is slightly worse for the 
LORAN hybridal (69%), considerably worse for the VOR (45%) and 
Air Data sensor hybrids (53%), and identical to the trajectory #1 results 
for the remaining configurations. Hence, the results from Trajectory 
#2 suggest that additional investigation into trajectory effects on VOR's 
and Air Data Sensor's performance may be necessary. 

Selecting Navigation Strategies Using the NSM Expert 

System 

The NSM-Recommended Navaid strategies were compared with 
covariance-detestnined strategies for various navigation sensor suite 


scenarios [9,12]. For example, the NSM Expert was asked to find the 
best two-measurement hybrid strategies for the following navaid 
sensors: 

Sensor Scenario #1 = [2 GPS satellites, LORAN Station A, 

DME Station A, VOR Station C, Doppler Radar) 

The results obtained by mixing combinations of the Scenario #1 
sensors are shown in Fig. 6. The NSM Expen correctly identified the 
best, next-best, and third-best navigation strategies. Additional NSM 
Expen results are given elsewhere [9,12]. 



* i v 

0 10 20 30 40 50 60 


Turn, mmoles 

Figure 6. . NSM-Recommended Navaid Strategies 


In addition to recommending navigation strategies from a given set 
of sensors, several heuristic sensor selection schemes were determined 
from the ANOVA results, the Scheffe comparisons, and the ED3 
decision trees. 

For single-type hybrids whose ground stations are at or near the 
same location, the ranking in order of best to worst performance was 
determined from ANOVA and the Scheffe tests as follows: 

Ranking heuristic for — [GPS, LORAN or TACAN, DME, VOR} 
co-located navaids 
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The ranking scheme above applies to single-station hybrids (or double- 
satellite GPS hybrids). The Scheffe comparisons show that a TACAN 
or LORAN selection depends on trajectory effects: this means that 
either a TACAN or LORAN hybrid could provide stadstically lower 
RSS position errors depending on the aircraft flight path with respect to 
the stations being compared. The ranking heuristic is accurate with a 
99% confidence level. If non-colocated stations are compared, trajec- 
tory effects must be considered for these single-station hybrids. The 
ANOVA and Scheffe comparison tests showed that double- station 
hybrids provide much smaller RSS position errors than single-station 
hybrids [9,1 1,12]. The performance difference between two and three 
stations is not as dramatic but is nonetheless statistically significant 
[ 12 ]. 

For multi-station hybrids of the same navaid type, the ANOVA and 
ID3 results provide the following ranking scheme from best to worst 
navigation performance as follows [9]: 

Ranking heuristic for 

multi-station hybrids = (4 GPS, 3 GPS, 3 LORAN, 2 LORAN, 
and on-board sensor 3 TACAN, 2 TACAN, 3 DME, 2 DME, 
hybrids Doppler Radar, Air data sensor, 3 VOR, 

2 VOR) 

However, the decision trees highlight the importance of trajectory 
effects in RSS position error estimation. 

CONCLUSIONS 

This paper summarized two research efforts to develop expen 
systems using systematic methods: The design methodology for 
multiple cooperating rule-based systems, and the development of a 
navigation sensor management system. Nine modular rule-based 
systems, collectively called AUTOCREW, were designed to automate 
functions and decisions associated with a combat aircraft's 
subsystems. The nine AUTOCREW knowledge bases were designed 
individually and areas of cooperation between the knowledge bases 
were identified. Each knowledge-base was designed using a graphical 
symbology that clearly illustrated the relationships between functional 
and decision-making tasks. Performance metrics were developed to 
evaluate the workload of each knowledge base and to assess the 
cooperation between the rule-bases. 

The NAVIGATOR'S sensor management task was treated in detail. 
The performances of seven navigation systems aiding a medium- 
accuracy Inertial Navigation System (INS) were investigated using 
Kalman Filter covariance analyses. A Navigation Sensor Management 
Expert System was formulated from covariance simulation data using 
the Analysis-of-Variance (ANOVA) method and the ED3 algorithm. 
ANOVA results show that statistically different position accuracies are 
obtained when different navaids are used, the number of navaids aiding 
the INS is varied, the aircraft's trajectory is varied, and the 
performance history is varied. The ED3 algorithm determines the NSM 
Expert's classification "rules" in the form of decision trees. The 
performances of these decision trees were assessed on two arbitrary 
trajectories, and the results demonstrate that the NSM Expert adapts to 
new situations and provides reasonable estimates of the expected 
hybrid performance. The NSM Expert demonstrates that carefully- 
planned simulation experiments can be used to develop a fully- 
operational expert system with a designer-specified performance 
effectiveness. 
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